Representation Is Representation of Similarities Representation Is Representation of Similarities

Advanced perceptual systems are faced with the problem of securing a principled relationship between the world and its internal representation. I propose a uniied approach to visual representation, based on Shepard's (1968) notion of second-order isomorphism. According to the proposed theory, a shape is represented internally by the responses of a few tuned modules, each of which is broadly selective for some reference shape, whose similarity to the stimulus it measures. The result is a philosophically appealing, computationally feasible, biologically credible, and formally veridical representation of a distal shape space. This approach supports representation of and discrimination among shapes radically diierent from the reference ones, while bypassing the need for the computationally problematic decomposition into parts; it also addresses the needs of shape categorization, and can be used to derive a range of models of perceived similarity. Abstract Advanced perceptual systems are faced with the problem of securing a principled (ideally, veridical) relationship between the world and its internal representation. I propose a uniied approach to visual representation, addressing both the needs of superordinate and basic-level categorization and of identiication of speciic instances of familiar categories. According to the proposed theory, a shape is represented internally by the responses of a small number of tuned modules. Each of these modules is broadly selective to some reference shape, whose similarity to the stimulus it measures. This amounts to embedding the stimulus in a low-dimensional proximal shape space, spanned by the outputs of the active modules. This shape space supports representation of distal shape similarities which is veridical in the sense of Shepard's (1968) notion of second-order isomorphism (i.e., of correspondence between distal and proximal similarities among shapes, rather than between distal shapes and their proximal representations). Representation in terms of similarities to reference shapes supports processing (e.g., discrimination) of shapes radically diierent from the reference ones, while bypassing the need for the computation-ally problematic decomposition into parts, postulated by other theories. Furthermore, a general expression for similarity between two stimuli, based on comparisons to reference shapes, can be used to derive models of perceived similarity ranging from continuous, symmetric, and hierarchical , as in the multidimensional scaling models (Shepard, 1980), to discrete and non-hierarchical,